Minimization of Continuous Bethe Approximations: A Positive Variation
نویسندگان
چکیده
We develop convergent minimization algorithms for Bethe variational approximations which explicitly constrain marginal estimates to families of valid distributions. While existing message passing algorithms define fixed point iterations corresponding to stationary points of the Bethe free energy, their greedy dynamics do not distinguish between local minima and maxima, and can fail to converge. For continuous estimation problems, this instability is linked to the creation of invalid marginal estimates, such as Gaussians with negative variance. Conversely, our approach leverages multiplier methods with well-understood convergence properties, and uses bound projection methods to ensure that marginal approximations are valid at all iterations. We derive general algorithms for discrete and Gaussian pairwise Markov random fields, showing improvements over standard loopy belief propagation. We also apply our method to a hybrid model with both discrete and continuous variables, showing improvements over expectation propagation.
منابع مشابه
Bethe Free Energy and Contrastive Divergence Approximations
Bethe Free Energy and Contrastive Divergence Approximations for Undirected Graphical Models Yee Whye Teh Doctorate of Philosophy Graduate Department of Computer Science University of Toronto 2003 As the machine learning community tackles more complex and harder problems, the graphical models needed to solve such problems become larger and more complicated. As a result performing inference and l...
متن کاملConvexity Arguments for Efficient Minimization of the Bethe and Kikuchi Free Energies
Loopy and generalized belief propagation are popular algorithms for approximate inference in Markov random fields and Bayesian networks. Fixed points of these algorithms have been shown to correspond to extrema of the Bethe and Kikuchi free energy, both of which are approximations of the exact Helmholtz free energy. However, belief propagation does not always converge, which motivates approache...
متن کاملBounds on the Bethe Free Energy for Gaussian Networks
We address the problem of computing approximate marginals in Gaussian probabilistic models by using mean field and fractional Bethe approximations. As an extension of Welling and Teh (2001), we define the Gaussian fractional Bethe free energy in terms of the moment parameters of the approximate marginals and derive an upper and lower bound for it. We give necessary conditions for the Gaussian f...
متن کاملThe Estimation of Distributions and the Minimum Relative Entropy Principle
Estimation of Distribution Algorithms (EDA) have been proposed as an extension of genetic algorithms. In this paper we explain the relationship of EDA to algorithms developed in statistics, artificial intelligence, and statistical physics. The major design issues are discussed within a general interdisciplinary framework. It is shown that maximum entropy approximations play a crucial role. All ...
متن کاملThe Bethe, Brown, and Stehn Lamb Shift Calculation Revisited
As applied by Bethe, Brown, and Stehn in 1950, the first theory of the Lamb shift, although it contains many physical and mathematical approximations, is known to be in agreement with experiment (tens of MHz discrepancies). In the present work, thanks to new analytic formulas and computer means, we perform their original method of calculation again, without two approximations in the contributio...
متن کامل